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Abstract 

Within  net-centric  operations,  military  commanders  need  to  plan  and  assess  operations  in  the 
context  of  a  “complex  endeavor.”  Potential  courses-of-action  (COA)  must  take  into  account 
impacts  dealing  with:  political,  diplomatic,  social,  economic  as  well  as  military,  information  and 
infrastructure.  When  considering  cyber  operations,  military  timelines  have  been  reduced  by 
several  orders  of  magnitude  to  keep  pace  with  operations  being  conducted  within  a  global 
cyberspace  environment.  Suggested  is  a  0th  -  9th  order-effects  taxonomy  to  give  a  more  robust, 
complete  and  consistent  “effects”  analysis  of  potential  COAs.  This  taxonomy  can  be  sub-divided 
into  three  groups:  1)  “tactical  level  effects;”  local  area  around  the  target;  2)  “operational  level 
effects;”  region  surrounding  the  target;  and,  3)  “strategic  level  effects;”  global  nature  of  the 
target.  Of  note  is  that  effects  within  the  cyber  domain  can  achieve  “global  effects”  far  faster  than 
other  domains  due  to  its  unique  “borderless”  nature.  For  each  “layer”  of  effectiveness,  metrics 
are  used  to  “assess/plan”  when  considering  the  “desired  effects”  within  a  complex  endeavor. 
Performing  analysis  on  nth  order-effects  is  primarily  qualitative;  therefore  a  “subjective  logic” 
approach  using  belief,  disbelief,  and  uncertainty  as  sub-divisions  was  used  to  evaluate: 
Detrimental  to  Operations,  Unacceptable,  Acceptable,  Very  Acceptable,  and  Significantly 
Acceptable  to  Operations. 

Introduction 

At  issue  today  is  how  to  measure  the  effectiveness  (or  non-effectiveness)  of  particular  event(s) 
that  occur  during  military  engagements  (integration  of  air,  space  and  cyberspace).  What  is  even 
more  challenging  is  to  determine  metrics  for  Nth-Order  effects;  not  to  mention  that  there  is  no 
agreed  upon  definition  for  an  Nth-Order  effect.  It  is  no  surprise  that  cyber  operations  have 
become  co-equal  with  other  operations  in  importance  and  criticality.  In  2006,  the  Secretary  of  the 
Air  Force  added  a  third  arm  to  the  Air  Force  Mission:  that  of  cyberspace.  Given  the  growing 
importance  of  the  role  of  cyberspace  within  the  Air  Force,  a  sound  strategy  for  achieving 
reliable,  survivable,  assured  and  continuous  cyber  operations  has  become  paramount  within  the 
21st  century.  In  fact,  cyber  operations  have  the  potential  to  become  an  influential  power 
provided  by  the  Air  Force.  Cyber  operations  can  be  thought  of  as  the  third  leg  of  a  “C2  Triad” 
(Air  and  Space  being  the  other  two).  By  adding  this  third  leg,  fundamental  methodologies,  such 
as  Effects  Based  Operations  (EBO)  can  be  greatly  enhanced. 

The  networks  and  information  systems  that  are  being  constructed  today  are  extremely  complex. 
An  adversary  cyber  attack  against  a  network  could  have  cascading  and  devastating  effects  on 
other  portions  of  the  information  enterprise.  To  defend  against  a  network  wide  attack,  it  is 
imperative  that  we  know  what  is  on  our  systems  and  their  composition.  We  need  to  know  what 
computers  we  have,  what  applications  are  running,  what  vulnerabilities  exist  and  what  networks 
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are  related  to  other  networks.  Only  when  anned  with  this  type  of  information  can  we  possibly 
adapt  our  countermeasures  and  protection  procedures  to  counter  an  adversary. 

When  considering  cyber  operations,  military  timelines  will  have  to  be  reduced  by  several  orders 
of  magnitude  in  order  to  be  able  to  keep  pace  with  operations  being  conducted  within  a  global 
cyberspace  environment.  Timelines  to  conduct  a  military  operation  have  significantly  been 
reduced  from  weeks-months  to  minutes-seconds.  For  example,  in  the  mid-1990s,  the  then  CSAF 
Gen  Jumper  embarked  on  the  goal  of  performing  “sensor-to-shooter”  operations  in  “single-digit- 
minutes.”  This  was  no  easy  task.  The  inter-relationships  of  data,  information,  awareness, 
understanding  and  decision  making  along  with  the  corresponding  actions  is  quite  complex. 
Imagine  conducting  a  “sensor-to-shooter”  operation  from  when  a  sensor  “detects”  a  target  (to)  to 
when  the  target  is  neutralized  (tn);  all  this  has  to  occur  within  “single-digit-minutes”  or  a 
maximum  of  599-seconds!  Since  multiple  “entities”1  are  involved  there  are  different  timelines 
for  each  entity;  thus  to  achieve  “shared  awareness”  or  “shared  understanding”  one  has  to  wait  for 
the  “slowest  entity.” 

Within  the  context  of  net-centric  operations,  there  is  a  series  of  timelines  that  one  needs  to 
consider  when  conducting  operations.  Appendix  A  provides  a  graphic  and  a  table  illustrating 
these  relationships. 

Now  consider  today,  where  the  “sensor-to-shooter”  resides  in  the  cyber  domain.  This  timeline 
needs  to  be  conducted  in  milli-seconds-to-seconds-to-minutes!  These  are  the  fundamental  issues 
that  need  to  be  considered  when  dealing  with  net-centric  operations  in  the  composite  of  the  three 
domains  (air,  space,  cyber). 

Phases  of  Conflict 

The  Phases  of  Conflict  (Figure  1)  that  are  used  typically  are  applied  to  the  air  domain  of  military 
operations.  However,  military  engagements  must  take  into  account  an  integrated  view  of  air, 
space  and  cyberspace  operations.  For  example,  Figure  1  illustrates  that  within  air,  space  and 
cyberspace  the  AF  goals  of  Global  Vigilance,  Global  Reach,  and  Global  Power  have  a  place  in 
all  the  stages.  Naturally,  the  levels  of  engagement  will  be  different  for  each  of  the  seven  phases. 
For  example,  cyber  engagements  could  be  higher  during  the  periods  of  peace  and  crisis  and  act 
as  neutralizers  with  the  goal  of  not  moving  into  Operations  Other  than  War  (OOTW)  or  war 
itself.  During  recovery  and  reconstitution,  military  engagements  will  be  more  defensive  in  nature 
to  restore  and  recover  military  air,  space  and  cyberspace  systems. 


1  An  entity  can  be  an  individual,  a  team,  an  organization  or  a  software  algorithm,  depending  on  the  application. 
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Figure  1:  Phases  of  Conflict 

Some  examples  of  the  application  within  the  cyber  domain: 

a)  Global  Vigilance:  Utilizing  the  cyber  domain  provides  a  significant  ability  to 
continuously  monitor  world  events  and  to  focus  on  anything,  anywhere  and  at  anytime. 
Extending  Global  Vigilance  into  the  cyber  domain  can  perform  missions  in  all  the 
phases;  however,  the  focus  differs  depending  on  the  phase  of  conflict. 

b)  Global  Reach:  Extending  into  the  Cyber  Domain,  the  use  of  Global  Reach  can  perform 
missions  in  all  phases.  Naturally,  the  intensity  changes  depending  on  the  phase.  One 
mission  might  be  to  “follow  the  money”  of  a  suspected  adversary.  The  result  would  be  to: 
1)  freeze  the  assets,  and,  2)  find  the  adversary  (anywhere,  anytime).  The  goal  would  be  to 
minimize  the  chances  of  escalating  to  the  next  phase.  Global  Reach  can  be  applied  to 
defensive  cyber  operations  as  well.  This  would  entail  the  ability  to  detect  an  intrusion 
(Global  Vigilance),  reach  out  and  touch  them  (Global  Reach)  and  then  negate  their 
actions  (Global  Power). 

c)  Global  Power:  Within  the  Cyber  Domain,  this  would  be  the  application  of  non-kinetic 
power  versus  kinetic  applications  (which  is  the  focus  of  the  current  definitions).  Global 
Power  can  be  applied  to  all  phases  of  conflict  in  the  form  of  D5  (Disrupt,  Degrade,  Deny, 
Deceive,  and  Destroy).  Naturally,  the  focus  changes  depending  on  the  phase.  In  OOTW 
and  War,  this  complements  the  traditional  kinetic  effects  to  achieve  the  desired  outcome. 
Global  Power  also  exists  in  defensive  operations  (which  are  applied  to  all  phases  of 
conflict)  as  the  ability  to  rapidly  negate  any  and  all  cyber  threats  in  conjunction  with 
Global  Reach  to  go  out  and  D5  an  adversary. 


2 

“  Data  does  not  represent  actual  values.  They  are  included  for  illustrated  purposes  only. 
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To  make  this  more  complex  within  the  21st  Century,  the  military  mission  space  encompasses  a 
wide  range  of  operations  (see  Figure  2),  which  includes  not  only  military  operations,  but  civil 
operations  as  well.  To  achieve  success  requires  1)  an  effects  based  approach  to  operations  where 
the  effects  that  need  to  be  considered  include  not  only  military  effects,  but  social,  political,  and 
economic  effects,  and  2)  the  ability  to  work  effectively  in  coalition  environments  that  include  not 
only  other  militaries  but  also  other  government  entities,  international  organizations,  businesses, 
and  a  variety  of  non-governmental  and  private  volunteer  organizations.  (Alberts,  2007:  2) 
Therefore,  a  cyber  engagement  is  analogous  to  either  an  air  or  space  engagement.  (Phister,  2011) 

Span  of  a  Complex  Endeavor 
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Span  of  a  Complex  Endeavor 

Figure  2:  Span  of  a  Complex  Endeavor 

The  work  that  was  accomplished  under  a  NATO  sponsored  “C2  Agility”  working  group  (SAS- 
065)  was  published  under  the  CCRP  Publication  umbrella  (Alberts,  2007:  122).  Figure  4 
summarizes  the  space  that  has  to  be  considered  when  dealing  with  a  complex  endeavor. 
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Figure  3:  Planning  in  the  Context  of  C2  and  Operations 


Conducting  operations  within  the  Cyber  Domain  is  complex;  especially  when  one  considers  the 
compressed  timelines  and  the  global  players  (military,  civil,  etc). 


Effects  Based  Operations 

Basically,  effects-based  operations  are  coordinated  sets  of  actions  directed  at  shaping  the 
behavior  of  friends,  foes,  and  neutrals  in  peace,  crisis,  and  war.  (Smith,  2002:  xiv)  The  concept 
of  effects-based  operations  is  a  broad  framework  which  includes  ideas  like  “nodal  targeting”  and 
“attrition-based  operations,”  but  which  offers  the  scope  and  flexibility  to  do  much  more:  to  look 
at  military  operations  in  peace,  crisis,  and  war  (plus  all  the  other  phases),  and  to  do  so  in  the 
context  of  a  cohesive  overall  national  political,  economic  and  military  effort.  (Smith,  2002:  104) 

A  key  factor  that  emerges  during  the  planning  and  assessment  portions  of  a  military  operation  is 
the  effects  that  can  be  achieved  (or  not  achieved)  given  the  stated  Command  intent.  Within  this 
military  concept,  effects-based  operations  translate  into  “What  are  the  effects  that  will  occur 
given  this  particular  COA  ?”  Additionally,  this  must  take  into  account  “Complex  Endeavors” 
since  the  Air  Force  does  not  “Fly,  Fight  and  Win”  singularly,  but  rather  in  a  multi-service, 
coalition  environment. 


3  COA  =  Course-Of-Action  or  Courses-Of- Action,  depending  on  the  context. 
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When  planning  within  a  complex  endeavor  the  effect  a  particular  action  will  have  must 
encompass  multiple  layers  from  the  target,  the  surrounding  area  all  the  way  up  to  national  policy. 
In  today’s  environment,  the  effects  on  national  policy,  say  from  collateral  damage,  far  outweigh 
the  military  significance  of  a  particular  target. 

Given  this  discussion,  exactly  what  is  meant  by  2nd  or  3ld  order-effects  given  a  particular  COA? 
There  is  no  clear  understanding  in  today’s  environment,  i.e.,  there  is  no  agreed-to  taxonomy  that 
attempts  to  categorize  what  is  a  2nd  Order-Effect,  or  a  3ld  Order-Effect,  or  even  an  Nth  Order- 
Effect  is  for  a  particular  COA.  This  paper  provides  a  suggestive  taxonomy  starting  with  a  0Ith 
order-effect  up  to  a  9th  Order-Effect. 

Proposed  Nth  Order  Taxonomy 

In  October  2002,  the  DoD’s  Command  and  Control  Research  Program  developed  the  “Code  of 
Best  Practice  for  C2  Assessment  (COBP)4  where  it  was  understood  that,  because  of  the 
complexity  of  C2  processes  and  systems,  analysis  in  this  area  requires  the  ability  to  understand 
how  Dimensional  Parameters  (DP),  Measures  of  Performance  (MoP),  Measures  of  C2 
Effectiveness  (MoCE),  Measures  of  Force  Effectiveness  (MoFE),  and  Measures  of  Policy 
Effectiveness  (MoPE)  are  linked  and  impact  on  one  another.  The  cumulative  set  of  these 
measures  is  denoted  as  Measures  of  Merit  (MoM)  in  the  COBP.  Determining  the  precise  nature 
of  these  relationships  nearly  always  proves  to  be  an  analytic  challenge  (NATO  COBP,  2002:  5). 
Appendix  B  provides  an  overview  of  the  metric  structure  developed  in  the  NATO  COBP  study. 

A  key  take-away  from  the  study  was  that,  within  net-centric  operations  one  has  to  consider  a 
hierarchy  of  effects  within  any  planning  environment  and  these  effects  need  to  consider 
operations  within  a  complex  endeavor  (Alberts,  2007:  122).  A  major  reason  for  this  is  that  a 
single  measure  does  not  provide  enough  detail  to  analyze  the  impact  of  specific  elements  (e.g., 
C2  system),  particularly  with  respect  to  second  and  third  order  effects  or  unintended 
consequences.  Unfortunately,  the  “NATO  COBP  for  C2  Assessments”  did  not  define  what  was 
meant  by  2nd  or  3ld  order  effects.  (NATO  COBP,  2002:  94) 

Taking  the  work  accomplished  in  the  NATO  COBP,  one  can  represent  Nth  order-effects  in  a 
hierarchical  form  starting  from  the  component  level  up  to  the  national  level.  Table  1  provides 
that  relationship  as  a  proposed  taxonomy. 


4  Document  stemmed  from  a  NATO  RTO  sponsored  Research  Group  (SAS-026)  and  was  published  under  the  Command  and 
Control  Research  Program  in  2002. 
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Table  1:  Nth  Order  Taxonomy  Relationships 


Nth 

Order 

Description 

Level 

Area 

MOPs/MOEs 

0 

Actual  Event 

Tactical 

Local 

1 

Desired  Effect 

Tactical 

Local 

DP 

2 

Collateral  Damage  (Physical) 

Tactical 

Local 

DP 

3 

Collateral  Damage  (Non-Physical) 

Tactical 

Local 

DP 

4 

C2  Systems  Effectiveness 

Operational 

Regional 

MoCE 

5 

Command  Intent  Effectiveness  (e.g., 

JFACC,  JFLCC,  JFMCC) 

Operational 

Regional 

MoFE 

6 

Command  Intent  (JTF)  Effectiveness 

Operational 

Regional 

MoFE 

7 

National  ROE/Policy  Effectiveness 

Strategic 

Global 

MoPE 

8 

Coalition  ROE/Policy  Effectiveness 

Strategic 

Global 

MoPE 

9 

International  ROE/Policy  Effectiveness 

Strategic 

Global 

MoPE 

From  Table  1,  there  are  three  groupings  within  the  proposed  taxonomy.  The  first  group  would  be 
the  “tactical  level  effects.”  This  would  encompass  the  local  area  around  the  particular  target.  The 
second  level  would  be  the  “operational  level  effects.”  This  would  be  analogous  to  the  region 
(e.g.,  the  Area  of  Responsibility — AOR  or  a  nation-state)  surrounding  the  particular  target.  The 
third  and  last  group  would  be  the  “strategic  level  effects.”  This  would  encompass  the  global 
nature  of  the  effects  around  the  particular  target5.  This  is  primarily  the  “policy  domain”  that 
would  take  into  account  effects  on  national,  coalition,  and  international  policy  given  a  particular 
COA. 

For  each  of  the  “layers”  of  effectiveness,  there  needs  to  be  a  methodology  for  the  metrics  that 
can  be  used  to  “assess”  and  “plan”  when  considering  the  “desired  effects”  for  mission  success 
within  a  complex  endeavor.  A  representative  sample  of  net-centric  factors/criteria  metrics  that 
would  be  applicable  can  be  found  in  the  AF  FY 12  Command  and  Control  Capabilities  Analysis 
Team  Final  Report  (a  comprehensive  list  can  be  found  in  Appendix  C.)  (AF  FY12  CAT,  2007: 
Appendix  B) 


5  Of  note  here  is  that  effects  within  the  cyber  domain  can  achieve  “global  effects”  far  faster  than  other  domains  due  to  its  unique 
“borderless”  nature.  For  example,  a  hacker  in  Country  A  can  significantly  impact  the  networks  in  Country  B;  without  ever 
leaving  Country  A. 
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0th  Order  Effect  (Actual  Event):  This  is  the  actual  “touching”  of  the  target.  The  methods  are 
different  depending  on  the  domain  (air,  space,  cyber)  being  utilized.  This  occurs  primarily  in  the 
physical  domain. 

l"  Order  Effect  (Desired  Effect):  Typically  this  can  be  characterized  as  “D5  Effects.”6  This  can 
occur  within  the  physical,  cognitive,  information  and  social  domains. 

2"cl  Order  Effect  (Collateral  Damage — Physical):  Typically  hardware  in  nature  (bridges,  roads). 
Represent  items  that  are  affected  that  were  not  supposed  to  be.  Can  also  be  non-combatants  who 
were  injured  in  the  event.  This  occurs  within  the  physical  domain. 

3rd  Order  Effect  (Collateral  Damage — Non-Physical):  Can  be  software  in  nature  but  can  also  be 
in  the  “Court  of  Public  Opinion”.  This  occurs  within  the  cognitive,  social  and  information 
domains. 

4th  Order  Effect  (C2  Systems  Effectiveness):  Covers  how  well  and  to  what  quality  were  the  CO  As 
developed,  selected  and  executed  towards  stated  goals  and  Command  Intent.  This  would  be  the 
effectiveness  of  an  AOC  as  a  whole  as  well  as  the  effectiveness  of  the  sub-C2  systems  contained 
within  an  AOC.  This  occurs  within  the  cognitive,  social,  information  as  well  as  physical 
domains. 

5th  Order  Effect  (JFACC  Command  Intent  Effectiveness) :  This  would  be  a  set  of  measures  as  to 
how  well  the  JFACC,  JFLCC,  JFMCC,  etc  (and  associated  staff)  developed  “Command  Intent.”7 
At  this  level  of  C2,  items  such  as  situational  awareness,  situational  understanding,  and  speed  of 
command  become  important  metrics  that  can  be  applied.  This  occurs  within  cognitive,  social  as 
well  as  the  information  domain. 

6th  order  Effect  (JTF  Command  Intent  Effectiveness):  This  would  be  a  set  of  measures  as  to  how 
well  the  JFC  (and  associated  staff)  developed  “Command  Intent.”  At  this  level  of  C2,  items  such 
as  joint  situational  awareness,  joint  situational  understanding,  and  speed  of  command  become 
important  metrics  that  can  be  applied.  This  occurs  within  cognitive,  social  as  well  as  the 
information  domain. 

7th  Order  Effect  (National  ROE/Policy  Effectiveness):  This  would  be  a  set  of  measures  as  to  how 
well,  at  the  National  Level,  the  military/political  leaders  (and  associated  staff)  developed  what 
could  be  termed  “National  Command  Intent.”  At  this  level  of  C2,  items  such  as  Rules  of 
Engagement  and  Policy  Statements  become  important  metrics  that  can  be  applied.  This  occurs 
within  cognitive,  social  as  well  as  the  information  domain. 


6  D5  =  Deny,  Deceive,  Destroy,  Disrupt,  and  Degrade. 

7  Initially  this  was  termed  “Commander’s  Intent”,  but  as  the  concept  of  Net-centricity  has  evolved  a  more  appropriate  tern  is 
“Command  Intent.” 
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8th  Order  Effect  (Coalition  ROE/P olicy  Effectiveness):  This  would  be  a  set  of  measures  as  to  how 
well,  at  the  Coalition  Level,  the  military/political  leaders  (and  associated  staff)  developed  what 
could  be  termed  “Coalition  Command  Intent.”  At  this  level  of  C2,  items  such  as  Rules  of 
Engagement  and  Policy  Statements  become  important  metrics  that  can  be  applied.  This  occurs 
within  cognitive,  social  as  well  as  the  information  domain. 

9th  Order  Effect  (International  ROE/Policy  Effectiveness):  This  would  be  a  set  of  measures  as  to 
how  well,  at  the  international  Level,  the  military/political  leaders  (and  associated  staff) 
developed  what  could  be  tenned  “International  Command  Intent.”  At  this  level  of  C2,  items  such 
as  Rules  of  Engagement  and  Policy  Statements  become  important  metrics  that  can  be  applied. 
This  occurs  within  cognitive,  social  as  well  as  the  information  domain. 

Nth-Order  Taxonomy  Examples 

There  are  two  major  principles  that  are  used  when  discussing  elements  of  national  power, 
namely  DIME  (Diplomatic,  Information,  Military,  and  Economic)  and  PEMSII  (Political, 
Economic,  Military,  Social,  Infrastructure,  and  Information).  The  sum  of  these  two  areas  can  be 
used  as  descriptors  to  attempt  to  measure  (either  quantitatively  or  qualitatively)  the  effect,  given 
a  set  of  potential  courses-of-action  (CO  A).  To  describe  a  Nth  order-effects  taxonomy,  a 
combination  of  the  two  principles  was  used  to  provide  a  basic  seven  item  system  of  descriptors 
for  the  0th  to  9th  order-effects,  namely  political,  diplomatic,  social,  economic,  military, 
information,  and  infrastructure.  Table  2  provides  some  top-level  examples  regarding  each  of  the 
Nth-orders. 
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Table  2:  Nth-Order  Taxonomy  Examples 


Order 

Title 

Political 

Diplomatic 

Social 

Economic 

Military 

Information 

Infrastructure 

0 

Actual  Event 

Nation-state  to 
Nation-state 

Impose 

“Sanctions” 

Impose 

“Beliefs” 

“’’Freezing” 

Assets 

Destroy, 

Degrade, 

Deceive,  Deny, 
Disrupt  a 

Military  Target 

Destroy,  Degrade, 
Deceive, 

Deny,  Disrupt  an 
Information  Target 

Destroy,  Degrade, 

Deceive, 

Deny,  Disrupt  an 
Infrastructure  Target 

' 

Desired  Effect 

Favorable 

outcome 

Favorable  outcome 

Favorable 

outcome 

Favorable 

outcome 

Favorable 

outcome 

Favorable  outcome 

Favorable  outcome 

2 

Collateral  Damage 
-  Physical 

Expelling  of  US 
Ambassador  or 
member  of 
embassy  staff 

Expelling  of  US 
Ambassador  or 
member  of 
embassy  staff 

Desired 
“beliefs” 
spilling  over 
into  other 
undesired  areas 

Others 
effected  with 
“Freezing 
Assets”  that 

was 

unintentional 

Destroy, 

Degrade, 

Deceive,  Deny, 
Disrupt  a 

Target 

unintentionally 

Destroy,  Degrade, 
Deceive,  Deny, 

Disrupt  a  Target 
unintentionally 

Destroy,  Degrade, 

Deceive,  Deny,  Disrupt  a 
Target  unintentionally 

3 

Collateral  Damage 
-  Non-Physical 

Seeking  “World 
Opinion”  that  is 
contrary  to 
desired  effects 

Retaliation  in 
another  area. 

Desired 
“beliefs” 
spilling  over 
into  other 
undesired  areas 

Others 
effected  with 
“Freezing 
Assets”  that 

was 

unintentional 

(e-g., 

emotional 

stress) 

Destroy, 

Degrade, 

Deceive,  Deny, 
Disrupt  a 

Target 

unintentionally 

Destroy,  Degrade, 
Deceive,  Deny, 

Disrupt  a  Target 
unintentionally 

Destroy,  Degrade, 

Deceive,  Deny,  Disrupt  a 
Target  unintentionally 

4 

C2  Systems 
Effectiveness 

System  does  not 
provide  nor 
incorporate 
robust  set  of 
political  options 
in  a  timely 
manner. 

System  does  not 
provide  nor 
incorporate  robust 
set  of  diplomatic 
options  in  a  timely 
manner. 

Fully 

comprehensive 
COA  that  takes 
into  account 
DIME  and 
PEMSII  effects 

Fully 

comprehensiv 
e  COA  that 
takes  into 
account  DIME 
and  PEMSII 
effects 

Fully 

comprehensive 
COA  that  takes 
into  account 
DIME  and 
PEMSII  effects 

Full  set  of 
capabilities  that 
provide  quality 
action-oriented 
information 

Provides  timely 
information  when  and 
where  needed 

5 

Command  Intent 

(JFACC) 

Effectiveness 

JFACC  does  not 
provide  nor 
incorporate 
robust  set  of 
options  in  a 
timely  manner. 

JFACC  does  not 
provide  nor 
incorporate  robust 
set  of  options  in  a 
timely  manner. 

JFACC  does 
not  provide  nor 
incorporate 
robust  set  of 
options  in  a 
timely  manner. 

JFACC  does 
not  provide 
nor 

incorporate 
robust  set  of 
options  in  a 
timely 
manner. 

JFACC  does 
not  provide  nor 
incorporate 
robust  set  of 
options  in  a 
timely  manner. 

JFACC  does  not 
provide  nor 
incorporate  robust  set 
of  options  in  a  timely 
manner. 

JFACC  does  not  provide 
nor  incorporate  robust  set 
of  options  in  a  timely 
manner. 

6 

Command  Intent 
(JFC)  Effectiveness 

JFC  does  not 
provide  nor 
incorporate 
robust  set  of 
political  options 
in  a  timely 
manner. 

JFC  does  not 
provide  nor 
incorporate  robust 
set  of  diplomatic 
options  in  a  timely 
manner. 

JFC  does  not 
provide  nor 
incorporate 
robust  set  of 
social  options 
in  a  timely 
manner. 

JFC  does  not 
provide  nor 
incorporate 
robust  set  of 
economic 
options  in  a 
timely 
manner. 

JFC  does  not 
provide  nor 
incorporate 
robust  set  of 
military 
options  in  a 
timely  manner. 

JFC  does  not  fully 
utilize  the 
capabilities  of  the 
information  provided 
to  develop  a  robust 
set  of  options  in  a 
timely  manner. 

JFC  does  not  fully  utilize 
the  capabilities  of  the 
available  infrastructure 
provided  to  develop  a 
robust  set  of  options  in  a 
timely  manner. 

7 

National 

ROE/Policy 

Effectiveness 

The  National 
level  does  not 
provide  nor 
incorporate 
robust  set  of 
political  options 
or  ROEs  in  a 
timely  manner. 

The  National  level 
does  not  provide 
nor  incorporate 
robust  set  of 
diplomatic  options 
or  ROEs  in  a 
timely  manner. 

The  National 
level  does  not 
provide  nor 
incorporate 
robust  set  of 
social  options 
or  ROEs  in  a 
timely  manner. 

The  National 
level  does  not 
provide  nor 
incorporate 
robust  set  of 
economic 
Options  or 

ROEs  in  a 

timely 

manner. 

The  National 
level  does  not 
provide  nor 
incorporate 
robust  set  of 
military 
options  or 

ROEs  in  a 
timely  manner. 

The  National  level 
does  not  fully  utilize 
the  capabilities  of  the 
information  provided 
to  develop  a  robust 
set  of  options  or 

ROEs  in  a  timely 
manner. 

The  National  level  does 
not  fully  utilize  the 
capabilities  of  the 
available  infrastructure 
provided  to  develop  a 
robust  set  of  options  or 
ROEs  in  a  timely 
manner. 

8 

Coalition 

ROE/Policy 

Effectiveness 

The  Coalition 
level  does  not 
provide  nor 
incorporate 
robust  set  of 
political  options 
or  ROEs  in  a 
timely  manner. 

The  Coalition  level 
does  not  provide 
nor  incorporate 
robust  set  of 
diplomatic  options 
or  ROEs  in  a 
timely  manner. 

The  Coalition 
level  does  not 
provide  nor 
incorporate 
robust  set  of 
social  options 
or  ROEs  in  a 
timely  manner. 

The  Coalition 
level  does  not 
provide  nor 
incorporate 
robust  set  of 
economic 
Options  or 

ROEs  in  a 

timely 

manner. 

The  Coalition 
level  does  not 
provide  nor 
incorporate 
robust  set  of 
military 
options  or 

ROEs  in  a 
timely  manner. 

The  Coalition  level 
does  not  fully  utilize 
the  capabilities  of  the 
information  provided 
to  develop  a  robust 
set  of  options  or 

ROEs  in  a  timely 
manner. 

The  Coalition  level  does 
not  fully  utilize  the 
capabilities  of  the 
available  infrastructure 
provided  to  develop  a 
robust  set  of  options  or 
ROEs  in  a  timely 
manner. 

9 

International 

ROE/Policy 

Effectiveness 

The 

International 
level  does  not 
provide  nor 
incorporate 
robust  set  of 
political  options 
or  ROEs  in  a 
timely  manner. 

The  International 
level  does  not 
provide  nor 
incorporate  robust 
set  of  diplomatic 
options  or  ROEs  in 
a  timely  manner. 

The 

International 
level  does  not 
provide  nor 
incorporate 
robust  set  of 
social  options 
or  ROEs  in  a 
timely  manner. 

The 

International 
level  does  not 
provide  nor 
incorporate 
robust  set  of 
economic 
Options  or 

ROEs  in  a 

timely 

manner. 

The 

International 
level  does  not 
provide  nor 
incorporate 
robust  set  of 
military 
options  or 

ROEs  in  a 
timely  manner. 

The  International 
level  does  not  fully 
utilize  the 
capabilities  of  the 
information  provided 
to  develop  a  robust 
set  of  options  or 

ROEs  in  a  timely 
manner. 

The  International  level 
does  not  fully  utilize  the 
capabilities  of  the 
available  infrastructure 
provided  to  develop  a 
robust  set  of  options  or 
ROEs  in  a  timely 
manner. 
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Mathematical  Representations  of  Nth-Order  Effects 

Given  the  proposed  nth  order-effects  taxonomy,  there  are  numerous  methods  of  performing 
analysis  to  determine  “effects”  on  “mission  effectiveness”  within  a  complex  endeavor. 

Since  perfonning  analysis  on  nth  order-effects  is  primarily  qualitative  (e.g.,  using  a  four-part 
evaluation:  totally-unacceptable,  unacceptable,  acceptable,  totally  acceptable),  using  “subjective 
logic”  (See  Appendix  D  for  a  short  overview)  is  a  possibility.  The  mathematical  representation 
can  be  shown  as: 


MOE(nth  Order)  =  f(belief)  +  f(disbelief)  +  f(uncertainty) 


This  would  entail  using  disbelief  uncertainty  algebra  for  nth  order  analysis.  (Denny,  2010:  Paper 
1 13)  Denny’s  paper  describes  a  methodology  that  can  translate  a  particular  nth  order  effect  into  a 
Measure  of  Effectiveness  (MOE).  The  scale  could  be  a  continuous  range  from  0  to  1  for  each  of 
the  variables  [belief  (b),  disbelief  (d)  and  uncertainty  (u)].  The  resultant  nth  order  effect  measure: 

0  =  Detrimental  to  Operations, 

.25  =  Unacceptable, 

.50  =  Acceptable, 

.75  =  Very  Acceptable, 

1  =  Significantly  Acceptable  to  Operations 


A  way  of  graphically  displaying  the  results  could  be  by  spider  graph.  Figure  4  illustrates  an 
example  for  comparing  two  different  Courses-of- Action  (COA)  options  for  a  particular  nth 
order-effect  (e.g.,  5th  Order-Effect:  Command  Intent  (JFACC)  Effectiveness).  As  shown,  COA-B 
option  provides  significantly  better  effectiveness  for  the  sub-items  under  the  5th  Order-Effect: 
Command  Intent  (JFACC)  Effectiveness. 

However,  to  determine  overall  mission  effectiveness,  for  example  at  the  campaign  level  (JTF),  a 
cross  order-effects  analysis  needs  to  be  performed.  This  would  provide  an  analysis  illustrating 
the  inter-relationships  of  the  effects  from  the  0th  to  9th  order.  For  each  of  the  nth  order-effects,  a 
watermelon  chart  could  be  used  to  collectively  show  how  the  effects  impact  the  operational 
mission  being  formed  as  a  complex  endeavor.  For  example,  if  an  analysis  is  being  performed  on 
the  5th  order  (Effectiveness  of  Command  Intent  of  the  JFACC),  Figure  5  illustrates  the  resultant 
analysis  where:  Cl  infrastructure,  C2=Information,  C3=Political,  C4=Diplomatic,  C5=Social, 
C6=Economic,  and  C7=Military.  As  shown,  weak  effectiveness  of  the  Military  (C7)  will  have  a 


Approved  for  Public  Release;  Distribution  Unlimited:  88ABW-2012-0423,  dated  27Janl2  Page  13 


Mission  Effectiveness:  Proposed  Nth  Order  Taxonomy 


catastrophic  effect  on  mission  while  robust  effectiveness  of  Economics  (C6)  will  have  a 
negligible  impact  on  mission  accomplishment. 


o 

Figure  4:  Spider  Graph  Representation  of  Effectiveness 


Catastrophic  Major  Substantial  Moderate  Minor  Negligible 


Impact  on  Mission 


Figure  5:  Watermelon  Representation  of  Impact  on  Mission* 9 


S  Data  does  not  represent  actual  values.  They  are  included  for  illustrated  purposes  only. 

9 

Data  does  not  represent  actual  values.  They  are  included  for  illustrated  purposes  only. 
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Figure  6:  Campaign  Level  Representation  of  Mission  Effectiveness10 

Figures  4,  5  and  6  are  just  samples  of  the  types  of  analytical  representations  that  can  be  formed 
to  illustrate  the  effects  for  each  of  the  nth  orders  into  a  single,  integrated  representation.  The  key 
is  to  incorporate  all  nine  layers  of  the  taxonomy  during  the  planning  and  assessment  phases  of 
any  operation.  This  is  a  more  robust  incorporation  of  effects-based  operations  to  provide  a  more 
comprehensive  view  of  potential  effects  given  a  set  of  COA. 


10  Data  does  not  represent  actual  values.  They  are  included  for  illustrated  purposes  only. 
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Conclusions 

Within  a  net-centric  operations  environment,  military  commanders  need  to  plan/assess 
operations  within  a  “complex  endeavor”  context.  Potential  COAs  must  incorporate  impacts  from 
areas  like:  political,  diplomatic,  social  and  economic  as  well  as  traditional  military,  information 
and  infrastructure  (i.e.,  DIME  and  PEMSII). 

A  suggested  methodology  is  a  0th  -  9th  order-effects  taxonomy  to  give  a  more  robust,  complete 
and  consistent  analysis  of  potential  COAs. 

Since  the  resultant  analysis  is  more  qualitative  than  quantitative,  a  “subjective  logic”  analytical 
approach  is  suggested.  This  approach  takes  into  account  qualitative  variables,  such  as  belief, 
disbelief,  and  uncertainty  when  applied  to  the  various  order-effects  list.  Consequently,  a  more 
robust  analysis  can  be  perfonned  on  potential  COAs. 

The  goal  is  to  select  better,  more  comprehensive  COAs  that  minimize  the  unacceptable  aspects 
and  maximize  the  effectiveness  of  mission  operations  within  a  complex  endeavor. 


Approved  for  Public  Release;  Distribution  Unlimited:  88ABW-2012-0423,  dated  27Janl2  Page  16 


Mission  Effectiveness:  Proposed  Nth  Order  Taxonomy 


Glossary 


Cyberspace 

A  global  domain  within  the  information  environment  consisting  of  the  interdependent 
network  of  information  technology  infrastructures,  including  the  Internet, 
telecommunications  networks,  computer  systems,  and  embedded  processors  and 
controllers. 

DoD  Joint  Pub  1-02,  page  86 

Global  Vigilance 

Ability  to  keep  an  unblinking  eye  on  any  entity  -  to  provide  warning  on  capabilities  and 
intentions,  as  well  as  identify  needs  and  opportunities. 

CSAF’s  Vector,  2008 

Global  Reach 

Ability  to  move,  supply  or  position  assets  -  with  unrivaled  velocity  and  precision 
anywhere. 

CSAF’s  Vector,  2008 

Global  Power 

Ability  to  hold  at  risk  or  strike  any  target,  anywhere,  and  project  swift,  frequently 
decisive,  precise  effects. 

CSAF’s  Vector,  2008 
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Appendix  A:  Operational  Timelines  and  Relationships 
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Figure  A-l:  Timeline  From  Data/Infonnation  (to)  to  Action  (tio) 


Timeline 

Relationship 

Definitions 

to 

t8 

ta8 

tb8 

talO 

tblO 

ta3-ta0  or  tb3-tb0 

Start  of  particular  time  reference 

Time  collaborative  decision  is  made 

Time  entity  A  decision  is  made 

Time  entity  B  decision  is  made 

Time  to  Action  for  entity  A 

Time  to  Action  for  entity  B 

Currency  of  the  data/information 

MAX(ta3,tb3)  -  tO 

MAX(ta5,tb5)  -  tO 

MAX(ta7,tb7)  -  tO 

MAX(ta9,tb9)  -  tO 
MAX(talO,tblO)  -  tO 

Currency  of  the  Shared  Information 

Currency  of  Shared  Awareness 

Currency  of  Shared  Understanding 

Currency  of  Shared  Decisions 

Currency  of  Shared  Actions 

Basic  Times 

ta2  -  taO,  tb2  -  tbO 
ta4  -  ta2,  tb4  -  tb2 
ta6  -  ta4,  tb6  -  tb4 
ta8  -  ta6,  tb8  -  tb6 

Time  to  receive,  process,  disseminate  data/information 

Time  to  analyze  and  gain  Awareness 

Time  to  analyze  and  gain  Understanding 

Time  to  analyze  and  make  a  decision  (Speed  of  Decision) 

Individual  Times 

ta4  -  taO,  tb4  -  tbO, 
ta6  -  taO,  tb6  -  tbO, 
ta8  -  taO,  tb8  -  tbO, 
talO  -  taO,  tblO  -  tbO 

Time  to  Awareness 

Time  to  Understanding 

Time  to  make  a  Decision  (Speed  of  Command) 

Time  to  Action  (Speed  of  Action) 

Collaboration  Times 

MAX(  ta4,tb4)  -  tO 
MAX(ta6,tb6)  -  tO 

MAX(ta8,tb8)  -  tO 
MAX(talO,tblO)  -  tO 

Time  to  Shared  Awareness  (time  to  achieve  a  Collective  Awareness) 

Time  to  Shared  Understanding  (time  to  achieve  a  Collective  Understanding) 
Time  to  make  a  Shared  Decision  (Joint  Speed  of  Decision) 

Time  to  Shared  Action  (Joint  Speed  of  Action) 

Table  A-l:  Net-Centric  Operations  Timeline  and  Relationships 
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Appendix  B:  Net-Centric  Operations  Levels  of  Metrics 

A  key  aspect  of  the  work  performed  under  the  NATO  COBP  endeavor  was  the  consideration  of  a 
hierarchy  when  dealing  with  effectiveness  in  a  net-centric  environment.  The  important  issues 
addressed  in  the  COBP  were  to  which  C2  performance  may  improve  force  and  policy 
effectiveness. 

Figure  B-l  provides  a  hierarchical  view.  The  column  on  the  left  indicates  the  level  and  the 
column  on  the  right  provides  an  example. 


Figure  B-l:  Hierarchical  View  of  Measures  of  Effectiveness  (NATO  COBP,  2002:  92) 


Where: 

MoM  =  Set  of  variables  that  focus  the  assessment  on  the  issues  of  interest. 

MoPE  =  focus  on  policy  and  societal  outcomes 

MoFE  =  focus  on  how  a  force  performs  its  mission  or  the  degree  to  which  it  meets  its  objectives 

MoCE  =  focus  on  the  impact  of  C2  systems  within  the  operational  context 

MoP  =  focus  on  internal  system  structure,  characteristics  and  behavior 

DP  =  focus  on  the  properties  or  characteristics  inherent  in  the  physical  C3  systems 
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Appendix  C:  Net-Centric  Factors/Criteria  Metrics 

The  metrics  below  is  a  combination  of  net-centric  metrics  that  have  been  used  to  date11. 


FACTORS 

CRITERIA 

CHARACTERISTIC  S/EXAMPEE  S 

Accessibility: 

data/information 

Percent  of  time  users  are  provided  with  (or  retrieve  from)  needed  products  under  various  loading  conditions. 

Accessibility: 

network 

Percent  of  time  a  network  is  available  for  use  by  users  to  provide  needed  products  under  various  loading  conditions. 

Accountability: 

individual 

decisions 

Measures  the  degree  to  which  individual  decisions  are  “accountable”  given  the  situation.  Metric  is  an  accountability  scale  in 
percent  (0%=not  accountable  of  individual  decisions  needed  and  what  is  available,  100%=max  accountability  between 
individual  decisions  needed  and  available. 

Accountability: 

collaborative 

decisions 

Measures  the  degree  to  which  collaborative  decisions  are  “accountable”  given  the  situation.  Metric  is  an  accountability  scale 
in  percent  (0%=not  accountable  of  collaborative  decisions  needed  and  what  is  available,  1 00%=max  accountability  between 
collaborative  decisions  needed  and  available. 

Accuracy: 
data  /  information 

Measure  of  error.  Metric  is  a  percent  scale  (0%=no  match  between  precision  level  needed  and  what  is  available,  1 00%=high 
degree  of  matching  between  precision  level  needed  and  available.  Examples:  avg  miss  distance:  +/-x  feet;  MHz:  +/-  x  hertz 

Accuracy:  shared 
data/information 

Measure  of  error  regarding  shared  information  between  entities.  Metric  is  a  percentage  scale  (0%=no  match  between  shared 
data/information  needed  and  what  is  available,  100%=high  degree  of  matching  between  shared  information  needed  and 
available. 

Accuracy:  shared 

awareness 

Measure  of  error  regarding  shared  awareness  between  entities.  Metric  is  a  percentage  scale  (0%=no  match  between  shared 
awareness  needed  and  what  is  available,  100%=high  degree  of  matching  between  shared  awareness  needed  and  available. 

Accuracy:  shared 
understanding 

Measure  of  error  regarding  shared  understanding  between  entities.  Metric  is  an  ordinal  scale  (0=no  match  between  shared 
understanding  needed  and  what  is  available,  10=high  degree  of  matching  between  shared  understanding  needed  and 
available. 

Availability: 

data/information 

Percent  of  time  users  are  provided  with  needed  products  under  various  loading  conditions.  Squadron,  wing,  base,  NAF, 
MAJCOM,  Air  Force  level,  Joint  level 

Availability: 
shared  awareness 

Percent  of  time  individuals  “share”  awareness. 

Availability: 

individual 

decisions 

Percent  of  time  individuals  “share”  decisions. 

Availability: 

collaborative 

decisions 

Percent  of  time  individuals  “share”  collaborative  decisions. 

Availability: 

individual 

understanding 

Percent  of  time  individuals  “share”  understanding. 

Availability: 

shared 

understanding 

Percent  of  time  individuals  “share”  understanding 

Awareness 

Awareness  is  a  process  state  existing  in  the  cognitive  domain.  It  takes  place  in  the  minds  of  key  leaders  and  their  supporting 
battlestaffs,  not  in  computers.  Awareness  is  achieved  through  a  complex  interaction  of  available  infonnation  (e.g.,  COP)  with 
prior  knowledge  and  beliefs  representing  the  experience  and  expertise  of  the  battlestaff.  Awareness  relates  to  the  operational 
situation  as  it  currently  is  or  was  in  the  past.  Human  perception  of  the  situation  as  it  is  and  as  it  is  becoming. 

Collective 

Awareness 

Collective  awareness  is  the  sum  of  the  elements  of  situational  awareness  held  by  all  the  actors  within  a  military,  interagency, 
or  coalition  structure. 

1 1  For  example,  the  AF  FY 12  C2  Capabilities  Analysis  Team’s  Final  Report  lists  these  unclassified  metrics  in  their  Appendix  B. 
These  metrics  are  from  open  sources  and  is  public  releasable. 
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FACTORS 

CRITERIA 

CHARACTERISTICS/EXAMPLES 

Collective 

Knowledge 

Degree  to  which  team  members  have  the  knowledge,  skills,  attributes,  and  abilities  that  they  need  to  accomplish  the  task  at 
hand 

Combat 

Assessment 

Battle  Damage  Assessment  (BDA)  +  Munitions  Effectiveness  Assessment  (MEA),  a  subset  of  which  is  Bomb  Impact 
Assessment  (BIA)  +  Mission  Assessment  (MA);  Re-attack  recommendation. 

Completeness: 
data  /  information 

Measures  the  completeness  of  the  data/information  provided.  Metric  is  the  percentage  of  relevant  data/information  received 
to  a  ground  truth  containing  all  the  data/information 

Completeness: 

shared 

data  /  information 

Measures  the  completeness  of  the  data/information  shared  between  entities.  Metric  is  the  percentage  of  relevant  shared 
data/information  received  to  a  ground  truth  containing  all  the  data/information 

Completeness: 
shared  awareness 

Measures  the  shared  awareness  between  entities.  Metric  is  the  percentage  of  relevant  shared  awareness  received  to  a  ground 
truth  containing  all  the  data/information 

Completeness: 

shared 

understanding 

Measures  the  shared  understanding  between  entities.  Metric  is  the  percentage  of  relevant  shared  understanding  received  to  a 
ground  truth  containing  all  the  data/information 

Computer 

Intrusion 

Detection 

Denial  of  service;  scanning  and  probing;  password  attacks;  privilege  grabbing;  hostile  code  insertion;  cyber  vandalism; 
proprietary  data  theft;  fraud,  waste  and  abuse;  audit  trail  tampering;  security  admin  attacks. 

Consistency: 

data/information 

Measures  the  degree  of  “deviation”  from  previous  data/information  gained  from  previous  time  period.  Measure  is  a 
percentage  deviation. 

Consistency: 

shared 

data  /  information 

Measures  the  degree  of  “deviation”  from  previous  data/information  shared  gained  from  previous  time  period.  Measure  is  a 
percentage  deviation. 

Consistency: 
shared  awareness 

Measures  the  degree  of  “deviation”  from  shared  awareness  gained  from  previous  time  period.  Measure  is  a  percentage 
deviation. 

Consistency: 

shared 

understanding 

Measures  the  degree  of  “deviation”  from  shared  understanding  gained  from  previous  time  period.  Measure  is  a  percentage 
deviation. 

Correctness: 

Organic 

Information 

Measure  to  determine  the  correctness  of  organic  information.  Metric  is  a  convergence  index  (0=no  correspondence  with 
ground  truth,  l=full  correspondence  with  ground  truth)  Data  matrix  comprised  of  relevant  information  items  estimates  (for 
instance:  detection,  ID,  velocity,  location,  heading,  etc.) 

Correctness: 

Shared 

Information 

Measures  the  correctness  of  the  data/information  that  is  shared  between  two  entities.  Metric  is  a  convergence  index  (0=no 
convergence,  l=full  convergence)  between  shared  information  and  ground  truth 

Correctness: 

Shared  Awareness 

Measures  the  level  of  shared  awareness  between  two  entities.  Metric  is  a  convergence  index  (0=no  awareness,  l=fiill 
awareness)  between  shared  awareness  and  ground  truth 

Correctness: 

Shared 

understanding 

Measures  the  level  of  shared  understanding  between  two  entities.  Metric  is  a  convergence  index  (0=no  awareness,  l=full 
awareness)  between  shared  understanding  and  ground  truth 

Currency: 

data/information 

Measures  the  age  of  the  data/information  from  the  time  it  was  originally  created 

Currency:  shared 
data  /  information 

Measures  the  age  (time  lag)  of  the  shared  data/information  from  the  time  it  was  originally  shared  between  entities 

Currency:  shared 

awareness 

Measures  the  age  (time  lag)  of  the  shared  awareness  from  the  time  it  was  originally  shared  between  entities 

Currency:  shared 
understanding 

Measures  the  age  (time  lag)  of  the  shared  understanding  from  the  time  it  was  originally  shared  between  entities 

Decision  Maker: 
Leadership 

Measures  the  ability  of  the  decision  maker  to  motivate  and  inspire  individuals  and  build  teams  to  achieve  mission  objectives. 
Metric  is  a  five  level  scale  (VL,  L,  M,  H,  and  VH)  of  a  decision  maker’s  leadership  capability. 

Decision  Maker: 

Confidence 

Measures  the  ability  of  the  decision  maker  to  gain  the  trust  of  superiors,  peers,  and  subordinates  by  demonstrating  integrity, 
professional  competence,  and  dedication  to  successfully  completing  the  current  mission.  Metric  is  a  five  level  scale  (VL,  L, 

M,  H,  and  VH)  of  a  decision  maker’s  confidence  factor. 
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FACTORS 

CRITERIA 

CHARACTERISTICS/EXAMPLES 

Decision  Maker: 

Balance 

Measures  the  ability  of  the  decision  maker  to  balance  personal  health  and  mental  well  being  with  the  demands  of  the  job  in 
order  to  stay  fresh,  alert,  and  effective.  Metric  is  a  five  level  scale  (VL,  L,  M,  H,  and  VH)  of  a  decision  maker’s  ability  to 
balance  numerous  factors  in  order  to  conduct  mission  operations. 

Decision  Maker: 

Decisiveness 

Measures  the  ability  of  the  decision  maker  to  provide  decisive  decisions  in  the  conduct  of  military  operations.  Metric  is  a 
continuous  level  scale  (0%=not  decisive  50%=somewhat  decisive,  100%=extremely  decisive). 

Decision  Maker: 
Adaptability 

Measures  the  ability  of  the  decision  maker  to  adaptive  to  withstand  or  adjust  to  changes  in  the  battlespace.  Metric  is  a 
continuous  level  scale  (0%=not  adaptable  50%=somewhat  adaptable,  1 00%=extremely  adaptable). 

Decision  Maker: 

Interpersonal 

Communications 

Skills 

Measures  the  interpersonal  communications  skills  of  the  decision  maker.  Metric  is  a  continuous  level  scale  (0%=little  to  no 
interpersonal  communications  skills,  50%=medium  interpersonal  communications  skills,  100%=high  degree  of  interpersonal 
communications  skills). 

Decision  Maker: 
Projection 

Measures  the  ability  of  a  decision  maker  to  conceptualize  future  actions  and  events  based  on  relevant  factors.  Metric  is  a 
continuous  level  scale  (0%=little  to  no  ability  to  project,  50%=medium  projection  skills,  100%=high  degree  of  projection 
skills). 

Decision  Maker: 

Multi-Tasking 

Ability 

Measures  the  ability  of  the  decision  maker  to  multi-task  to  effectively  manage  time  and  priorities  to  accomplish  multiple 
activities  simultaneously  within  the  battlespace.  Metric  is  a  continuous  level  scale  (0%=does  not  have  any  multi-tasking 
abilities,  50%=demonstrates  some  multi-tasking  abilities,  1 00%=exhibits  extreme  multi-tasking  abilities). 

Decision  Maker: 

Concentration 

Measures  the  ability  of  a  decision  maker  to  maintain  focus  and  deal  with  uncertainty  through  the  “fog  of  war”.  Metric  is  a 
continuous  level  scale  (0%=little  concentration  ability,  50%=medium  concentration  ability,  100%=high  degree  of 
concentration  ability). 

Decision  Maker: 
Negotiation 

Ability 

Measures  the  ability  of  the  decision  maker  to  tactfully  resolve  difficult  situations  when  internal  and  external  partners  disagree 
due  to  contrasting  opinions,  goals,  priorities,  methods,  and  /or  solutions.  Metric  is  a  five  level  scale  (VL,  L,  M,  H,  and  VH)  of 
the  decision  maker’s  negotiation  ability  or  skill. 

Decision  Maker: 
Courage 

Measures  the  decision  maker’s  ability  to  do  the  right  thing  at  the  right  time  in  spite  of  pressure  to  do  otherwise.  Includes  the 
ability  to  talk  about  doubt,  uncertainty,  and  bad  news.  Metric  is  a  continuous  level  scale  (0%=exhibits  little  to  no  courage, 
50%=exhibits  sufficient  courage,  1 00%=demonstrates  extreme  courage). 

Decision  Maker: 
Objectivity 

Measures  the  ability  of  the  decision  maker  to  clearly  look  at  the  operational  situations  (Blue,  Red,  Gray,  and  White)  as  they 
unfold  within  the  battlespace.  Metric  is  a  continuous  level  scale  (0%=exhibits  little  to  no  objectivity,  50%=exhibits  sufficient 
objectivity,  1 00%=demonstrates  extreme  objectivity). 

Decisions: 

Collaborative 

Accuracy 

Measures  the  degree  to  which  collaborative  decisions  are  “accurate”  given  the  situation.  Metric  is  a  percentage  scale  (0%=no 
match  between  collaborative  decisions  needed  and  what  is  available,  100%=high  degree  of  matching  between  collaborative 
decisions  needed  and  available. 

Decisions: 

Collaborative 

Adaptability 

Measures  the  ability  of  a  decision  maker  to  alter  collaborative  decisions  when  necessary  as  the  situation  changes.  Metric  is  a 
percentage  scale  (0%=cannot  adapt,  50%=  show  some  adaptability,  100%=shows  significant  adaptability) 

Decisions: 

Collaborative 

Appropriateness 

Measures  the  degree  to  which  collaborative  decisions  are  appropriate  given  the  situation.  Metric  is  a  percentage  scale 
(0%=collaborative  decision  not  appropriate  to  situation,  50%=collaborative  decision  may  or  may  not  be  appropriate  to 
situation,  100%=high  degree  of  appropriateness  between  collaborative  decisions  needed  and  available) 

Decisions: 

Collaborative 

Consistency 

Measures  the  degree  of  collaborative  decision  “consistency”.  Metric  is  a  percentage  index  that  relates  the  degree  of 
“deviation”  from  previous  collaborative  decisions  (0%=  no  consistency,  50%=some  consistency,  100%=maximum 
consistency) 

Decisions: 

Collaborative 

Completeness 

Measures  the  degree  of  decision  “completeness”.  Metric  is  the  percentage  of  individual  decision  relevant  to  the  situation  at 
hand  (0%=no  relevance,  50%=somewhat  relevant,  100%=maximum  relevancy) 

Decisions: 

Collaborative 

Currency 

Measures  the  time  to  make  a  collaborative  decision.  Metric  is  an  index  that  measures  the  time  it  takes  a  decision  maker  to 
make  a  collaborative  decision  given  a  situation. 

Decisions: 

Collaborative 

Flexibility 

Measures  the  ability  of  a  decision  maker  to  make  collaborative  decisions  in  different  situations.  Metric  is  a  percentage 
flexibility  scale  (0%=not  flexible,  50%=some  flexibility,  1 00%=significant  flexibility) 

Decisions: 

Collaborative 

Innovation 

Measures  the  ability  of  a  decision  maker  to  make  collaborative  decisions  in  new  ways  or  to  understand  new  things.  Metric  is 
a  percentage  scale  (0%=shows  no  innovation,  50%=shows  some  innovation,  100%=shows  significant  innovation) 
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FACTORS 

CRITERIA 

CHARACTERISTICS/EXAMPLES 

Decisions: 

Collaborative 

Mode  of  Decision 
Making 

Measures  the  controlling  nature  of  the  decision  maker  in  a  collaborative  situation.  Metric  is  a  percentage  scale  (0%=no 
control,  50%=some  control,  100%=total  control). 

Decisions: 

Collaborative 

Precision 

Measures  the  level  of  detail  of  a  particular  collaborative  decision  given  a  situation.  Metric  is  a  percentage  scale  (0%=no 
detail,  50%=some  details,  1 00%=significant  detail) 

Decisions: 

Collaborative 

Relevance 

Measures  the  degree  to  which  collaborative  decisions  are  “relevant”  given  the  situation.  Metric  is  a  percentage  collaborative 
relevance  scale  (0%=collaborative  decisions  not  relevant,  50%=collaborate  decisions  somewhat  relevant,  100%=high  degree 
of  relevancy  between  collaborative  decisions  needed  and  available. 

Decisions: 

Collaborative 

Responsiveness 

Measures  the  ability  of  a  decision  maker  to  make  effective  collaborative  decisions  given  a  situation.  Metric  is  a  percentage 
scale  (0%=does  not  make  effective  collaborative  decisions,  50%=makes  some  effective  collaborative  decisions,  100%=makes 
significant  effective  collaborative  decisions) 

Decisions: 

Collaborative 

Risk  Propensity 

Measures  the  decisions  makers  “collaborative  risk  taking”  ability  given  a  situation.  Metric  is  a  percentage  risk  level 
(0%=minimal  risk,  100%=maximum  risk)  or  risk  interval  (95%,  90%)  of  collaborative  decisions 

Decisions: 

Collaborative 

Robustness 

Measures  the  ability  of  a  decision  maker  to  use  levels  of  collaborative  decisions  across  a  range  of  missions  that  span  the 
spectrum  of  conflict.  Metric  is  a  percentage  scale  (0%=no  robustness,  50%=some  robustness,  1 00%=significant  robustness) 

Decisions: 

Collaborative 

Timeliness 

Measures  the  timeliness  to  which  a  decision  maker  makes  “collaborative  decisions”  given  a  situation.  Metric  is  a  percentage 
scale  (0%=does  not  make  collaborative  decisions  in  time  to  influence  an  outcome  of  a  given  situation,  1 00%=always  makes 
collaborative  decisions  in  time  to  influence  an  outcome  to  a  given  situation) 

Decisions: 

Collaborative 

Uncertainty 

Measures  the  uncertainty  level  of  a  collaborative  decision  given  a  situation.  Metric  is  a  percentage  confidence  scale 
(0%=uncertain,  100%=certain)  or  confidence  interval  (95%,  90%)  of  collaborative  decisions. 

Decisions: 

Individual 

Accuracy 

Measures  the  degree  to  which  decisions  are  “accurate”  given  the  situation.  Metric  is  a  percentage  accuracy  scale  (0%=no 
match  between  individual  decisions  needed  and  what  is  available,  50%=medium  match  between  individual  decisions  needed 
and  what  is  available,  100%=high  degree  of  matching  between  individual  decisions  needed  and  available. 

Decisions: 

Individual 

Adaptability 

Measures  the  ability  of  a  decision  maker  to  alter  individual  decisions  when  necessary  as  the  situation  changes.  Metric  is  a 
percentage  scale  (0%=cannot  adapt,  50%=  show  some  adaptability,  100%=shows  significant  adaptability) 

Decisions: 

Individual 

Appropriateness 

Measures  the  degree  to  which  decisions  are  appropriate  given  the  situation.  Metric  is  a  appropriate  scale  in  percentages 
(0%=individual  decision  not  appropriate  to  situation,  100%=high  degree  of  appropriateness  between  individual  decisions 
needed  and  available) 

Decisions: 

Individual 

Consistency 

Measures  the  degree  of  decision  “consistency”.  Metric  is  a  percentage  index  that  relates  the  degree  of  “deviation”  from 
previous  decisions  (0%=no  deviation,  50%=some  deviation,  100%=max  deviation). 

Decisions: 

Individual 

Completeness 

Measures  the  degree  of  decision  “completeness”.  Metric  is  the  percentage  of  individual  decision  relevant  to  the  situation  at 
hand 

Decisions: 

Individual 

Currency 

Measures  the  time  to  make  a  decision.  Metric  is  an  index  that  measures  the  time  it  takes  a  decision  maker  to  make  a  decision 
given  a  situation 

Decisions: 

Individual 

Flexibility 

Measures  the  ability  of  a  decision  maker  to  make  individual  decisions  in  different  situations.  Metric  is  a  percentage  flexibility 
scale  (0%=not  flexible,  50%=some  flexibility,  1 00%=significant  flexibility) 

Decisions: 

Individual 

Innovation 

Measures  the  ability  of  a  decision  maker  to  make  individual  decisions  in  new  ways  or  to  understand  new  things.  Metric  is  a 
percentage  scale  (0%=shows  no  innovation,  50%=shows  some  innovation,  100%=shows  significant  innovation) 

Decisions: 

Individual  Mode 

of  Decision 

Making 

Measures  the  controlling  nature  of  the  decision  maker.  Metric  is  a  percentage  scale  (0%=no  control,  50%=some  control, 
100%=total  control). 

Decisions: 

Measures  the  level  of  detail  of  a  particular  decision  given  a  situation.  Metric  is  a  percentage  scale  (0%=no  detail,  50%=some 
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Individual 

Precision 

details,  100%=significant  detail) 

Decisions: 

Individual 

Relevance 

Measures  the  degree  to  which  decisions  are  “relevant”  given  the  situation.  Metric  is  a  percentage  scale  (0%=individual 
decisions  not  relevant,  100%=high  degree  of  relevancy  between  individual  decisions  needed  and  available. 

Decisions: 

Individual 

Responsiveness 

Measures  the  ability  of  a  decision  maker  to  make  effective  individual  decisions  given  a  situation.  Metric  is  a  percentage  scale 
(0%=does  not  make  effective  decisions,  50%=makes  some  effective  decisions,  100%=makes  significant  effective  decisions) 

Decisions: 

Individual  Risk 
Propensity 

Measures  the  decisions  makers  “risk  taking”  ability  given  a  situation.  Metric  is  a  percentage  scale  measuring  the  risk  level 
(0%=minimal  risk,  100%=maximum  risk)  or  risk  interval  (95%,  90%)  of  individual  decisions 

Decisions: 

Individual 

Robustness 

Measures  the  ability  of  a  decision  maker  to  use  levels  of  decisions  across  a  range  of  mission  that  span  the  spectrum  of 
conflict.  Metric  is  a  percentage  scale  (0%=no  robustness,  50%=some  robustness,  100%=significant  robustness) 

Decisions: 

Individual 

Timeliness 

Measures  the  timeliness  to  which  a  decision  maker  makes  “decisions”  given  a  situation.  Metric  is  a  percentage  timeliness 
scale  (0%=does  not  make  decisions  in  time  to  influence  an  outcome  of  a  given  situation,  50%=makes  decisions  most  of  the 
time  to  influence  an  outcome  of  a  given  situation,  1 00%=always  makes  decisions  in  time  to  influence  an  outcome  to  a  given 
situation) 

Decisions: 

Individual 

Uncertainty 

Measures  the  uncertainty  level  of  a  decision  given  a  situation.  Metric  is  a  percentage  that  measures  confidence  scale 
(0%=uncertain,  1 00%=:certain)  or  confidence  interval  (95%,  90%)  of  individual  decisions. 

Effectiveness: 

Achievement  of 
Objectives 

Measures  the  degree  to  which  mission  objectives  are  achieved.  Metric  is  an  ordinal  scale  (0%=no  achievement, 
100%=maximum  achievement)  or  achievement  interval  (95%,  90%)  of  mission  objectives. 

Effectiveness: 

Agility 

Measures  the  ability  to  modify  forces  objectives  in  a  timely  manner.  Metric  is  an  ordinal  scale  (0=not  agile,  100=maximum 
agility) 

Effectiveness: 

Mission 

Measures  the  degree  to  which  a  force  accomplishes  its  assigned  military  mission.  It  is  multi-attributed.  These  metrics  exist 
largely  at  the  operational  level  and  below  when  thinking  inside  the  context  of  “traditional”  military  missions.  Metric  is 
percentage  of  mission  effectiveness. 

Effectiveness: 

Timeliness 

Measures  the  ability  to  achieve  a  mission  objective  in  a  timely  manner.  Metric  is  achieved  mission  objectives  divided  by  total 
mission  objectives  over  a  given  time  interval. 

Effectiveness: 

Efficiency 

Measures  the  ability  to  achieve  a  mission  objective  in  an  efficient  manner.  Metric  is  an  ordinal  scale  (0=not  efficient, 

1 00=maximum  efficiency) 

Extent:  degree  of 
data/information 

Measures  the  extent  of  shared  data/information 

Extent:  degree  of 
Shared  Awareness 

Measures  the  extent  of  shared  awareness 

Interaction: 

Individual 

Adaptability 

Measures  the  ability  to  alter  interactions  when  necessary  as  the  situation  changes.  Metric  is  a  percentage  scale  (0%=no 
adaptability,  50%=some  adaptability,  100%=significant  adaptability) 

Interaction: 

Individual 

Confidence 

Measures  the  state  of  being  certain.  Metric  is  a  percentage  index  of  confidence  in  the  individual  ranging  from  0%=no 
confidence  to  100%=total  confidence. 

Interaction: 
Individual  Latency 

Measures  the  time  lag  to  conduct  interactions  from  the  start  of  a  particular  situation.  Metric  is  a  time  interval  that  measures 
the  time  lag. 

Interaction: 
Individual  Quality 

Measures  the  quality  of  the  interactions  present  during  a  particular  situation.  Metric  is  a  percentage  scale  (0%=poor  quality, 
50%=medium  quality,  100%=high  quality) 

Interaction: 

Individual 

Quantity 

Measures  the  quantity  of  interactions  present  during  a  particular  situation.  Metric  is  a  percentage  scale  that  measures  the 
quantity  of  interactions  per  interval  of  time. 

Interaction: 

Individual  Reach 

Measures  the  end-to-end  distance  interaction  occurs.  Metric  are  a  percentage  of  nodes  (locations)  that  can  interact  in  desired 
access  modes. 

Interaction: 

Measures  the  state  of  being  certain.  Metric  is  a  percentage  index  of  confidence  in  the  organization  ranging  from  0%=no 
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Organization 

Confidence 

confidence  to  100%=total  confidence. 

Inter¬ 

cooperability: 

Organization-to- 

Organization 

The  ability  of  an  organization(s)  to  function  together  essentially  as  a  single  organization  to  essentially  achieve  shared 
understanding  among  each  other  and  to  use  the  information  exchanged  to  interact  effectively,  interdependently  and  adaptively 
toward  a  common  and  valued  set  of  goals.  The  metric  would  measure  the  level  of  inter-cooperability  (0-none,  50%-average, 

1 00%-maximum)  between  each  of  the  organizations  involved. 

Inter¬ 

cooperability: 

Individual-to- 

Individual 

The  ability  of  individual(s)  to  function  together  essentially  as  a  single  entity  to  essentially  achieve  shared  understanding 
among  each  other  and  to  use  the  information  exchanged  to  interact  effectively,  interdependently  and  adaptively  toward  a 
common  and  valued  set  of  goals.  The  metric  would  measure  the  level  of  inter-cooperability  (0-none,  50%-average,  1 00%- 
maximum)  between  each  of  the  individual  involved. 

Inter¬ 

cooperability: 

Individual-to-team 

The  ability  of  an  individual  and  teams  to  function  together  essentially  as  a  single  entity  to  essentially  achieve  shared 
understanding  among  each  other  and  to  use  the  information  exchanged  to  interact  effectively,  interdependently  and  adaptively 
toward  a  common  and  valued  set  of  goals.  The  metric  would  measure  the  level  of  inter-cooperability  (0-none,  50%-average, 

1 00%-maximum)  between  the  individual  and  team. 

Inter¬ 

cooperability: 

Individual-to- 

Organization 

The  ability  of  individual(s)  and  organizations  to  function  together  essentially  as  a  single  entity  to  essentially  achieve  shared 
understanding  among  each  other  and  to  use  the  information  exchanged  to  interact  effectively,  interdependently  and  adaptively 
toward  a  common  and  valued  set  of  goals.  The  metric  would  measure  the  level  of  inter-cooperability  (0-none,  50%-average, 

1 00%-maximum)  between  the  individual  and  organization. 

Inter¬ 

cooperability: 

Team-to- 

Organization 

The  ability  of  a  team  and  organization  to  function  together  essentially  as  a  single  entity  to  essentially  achieve  shared 
understanding  among  each  other  and  to  use  the  information  exchanged  to  interact  effectively,  interdependently  and  adaptively 
toward  a  common  and  valued  set  of  goals.  The  metric  would  measure  the  level  of  inter-cooperability  (0-none,  50%-average, 

1 00%-maximum)  between  the  team  and  organization. 

Inter¬ 

cooperability: 

Team-to-Team 

The  ability  of  teams  to  function  together  essentially  as  a  single  entity  to  essentially  achieve  shared  understanding  among  each 
other  and  to  use  the  information  exchanged  to  interact  effectively,  interdependently  and  adaptively  toward  a  common  and 
valued  set  of  goals.  The  metric  would  measure  the  level  of  inter-cooperability  (0-none,  50%-average,  1 00%-maximum) 
between  teams. 

Maintainability: 

data/infonnation 

Measures  the  maintainability  of  the  data/infonnation.  Metric,  depicted  as  a  percentage,  is  the  ease  to  which  the 
data/information  is  maintained  within  the  specified  system  (0%=not  maintainable  by  pre-defined  standards,  50%=somewhat 
maintainable  by  pre-defined  standards,  100%=totally  maintainable  within  pre-defined  standards). 

Maintainability: 

system 

Measures  the  maintainability  of  a  particular  system.  Metric,  depicted  as  a  percentage,  is  the  ease  to  which  the  system  is 
maintained  within  pre-defined  standards  -  e.g.,  equipment  accessibility,  shop  replaceable  unit,  line  replaceable  units,  and 
depot  level  repair.  (0%=not  maintainable  by  pre-defined  standards,  50%=somewhat  maintainable  by  pre-defined  standards, 
100%=totally  maintainable  within  pre-defined  standards). 

Maintainability: 

network 

Measures  the  maintainability  of  a  particular  network.  Metric,  depicted  as  a  percentage,  is  the  ease  to  which  the  network  is 
maintained  within  pre-defined  standards  -  e.g.,  equipment  accessibility,  shop  replaceable  unit,  line  replaceable  units,  and 
depot  level  repair.  (0%=not  maintainable  by  pre-defined  standards,  50%=somewhat  maintainable  by  pre-defined  standards, 
100%=totally  maintainable  within  pre-defined  standards). 

Network  Agility 

Measures  the  ability  to  modify  an  entire  network  un  a  timely  manner.  Metric  is  a  percentage  rating  of  agility  (0%  =  no  agility, 
50%  =  medium  agility,  100%=maximum  agility) 

Network 

Assurance 

Measures  the  security  of  an  entire  network.  Metric  is  a  percentage  rating  of  network  security  (100%  =  highly  secure,  90%  = 
secure,  0%  =  not  secure)  based  on  network  and  node  encryption  levels,  type  of  security  management  systems  provided,  etc. 

Network 

Availability 

Measures  the  time  all  authorized  users  have  access  to  the  network.  This  is  necessary  if  current  information  is  to  be  shared  and 
if  the  user  community  is  to  develop  trust  and  confidence  in  using  the  information  in  the  system.  Metric  is  percentage  of  time 
network  is  available  to  users. 

Network  Reach 

Measures  the  end-to-end  extent  (or  reach)  of  the  network.  Metric  is  the  percent  of  nodes  that  can  communicate  in  desired 
access  modes,  information  formats,  and  applications 

Network  Richness 

Measures  the  quality  and  breath  of  the  information  found  in  the  network.  Metric  is  a  percentage  scale  (0%=not  rich,  50%= 
some  richness,  100%=maximum  richness)  or  interval  scale  (95%,  90%)  of  network  richness. 

Network 

Reliability 

Measures  the  network’s  ability  to  consistently  produce  the  same  results.  Metric  is  a  percentage  scale  (0%=not  reliable,  50%= 
somewhat  reliable,  100%=maximum  reliability)  or  interval  scale  (95%,  90%)  of  network  reliability. 

Precision: 

data  /  information 

Measures  the  level  of  measurement  detail  of  a  data/infonnation  item.  For  example,  Measure  of  repeatability,  probability  of 
damage/kill  (Pd/Pk) 

Precision:  shared 

data/information 

Measures  the  level  of  granularity  of  a  shared  data/information  item.  Measure  is  percentage  deviation  from  actual  “truth”,  for 
example  Frequency  +-  5%. 
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Precision:  shared 

awareness 

Measures  the  level  of  granularity  of  shared  awareness. 

Measure  is  percentage  deviation  from  actual  “truth”,  for  example,  9/10  commanders  have  the  same  awareness  equates  to  90% 
of  shared  awareness. 

Precision:  shared 
understanding 

Measures  the  level  of  granularity  of  shared  understanding.  Measure  is  percentage  deviation  from  actual  “truth”,  for  example, 
9/10  commanders  have  the  same  understanding  equates  to  90%  of  shared  understanding. 

Relevance: 

data/information 

Measures  the  proportion  of  information  collected  that  is  related  to  the  task  at  hand 

Relevance:  shared 

data  /  information 

Measures  the  proportion  of  shared  information  collected  that  is  related  to  the  task  at  hand.  Metric  is  a  percentage  scale  that 
measures  the  relevance  of  the  shared  data/information  (0%=no  relevance,  50%  =  some  relevance,  100%=maximum 
relevance) 

Reliability: 

data/information 

Information  -  trusted/proven  source,  new/unproven  source;  Computer  -  Mean-Time  Between  Failures  (MTBF),  ID  faults  to 
board  level  95%  accuracy; 

Reliability:  system 

The  probability  a  system  will  perform  satisfactorily  for  a  period  of  time  under  a  set  of  conditions.  Metric  is  a  percentage  scale 
(0%=not  reliable,  50%=somewhat  reliable,  100%=highly  reliable). 

Reliability: 

individual 

decisions 

Measures  the  reliability  of  the  decisions  made  by  an  individual  over  a  period  of  time  under  a  set  of  conditions.  Metric  is  a 
percentage  scale  (0%=not  reliable,  50%=somewhat  reliable,  100%=highly  reliable) 

Reliability: 

collaborative 

decisions 

Measures  the  reliability  of  collaborative  decisions  made  by  a  team  of  individuals  over  a  period  of  time  under  a  set  of 
conditions.  Metric  is  a  percentage  scale  (0%=not  reliable,  50%=somewhat  reliable,  100%=highly  reliable) 

Robustness 

Ability  to  maintain  effectiveness  across  a  range  of  tasks,  situations,  and  conditions  across  a  range  of  missions  that  span  the 
spectrum  of  conflict.  Metric  would  be  a  percentage  scale  (0%=no  robustness,  50%=some  robustness,  100%=maximum 
robustness) 

Shared  Awareness 

Shared  awareness  is  the  human  perception  of  the  situation  as  it  is  and  as  it  is  becoming.  The  elements  of  military  situations 
include:  physical  environment,  the  capabilities  and  intentions  of  red,  blue,  and  other  forces  and  effectors,  and  the  political, 
military,  social,  economic,  and  information  contexts. 

Shared 

Understanding 

Shared  understanding  is  the  recognition  of  patterns,  cause  and  effect  relationships,  dynamic  futures,  and  opportunities  and 
risks  that  are  shared  between  individuals,  organizations,  or  other  entities. 

Situational 

Awareness 

Situational  awareness  is  the  “who’s”,  “where”  category.  It  includes  friendly,  enemy,  and  neutrals  location,  status; 
vulnerabilities  and  capabilities.  It  also  includes  weather  and  terrain  features.  For  targeting  it  includes  detect,  locate,  ID,  track, 
and  display. 

Situational 

Understanding 

Situational  understanding  is  the  “what  does  it  mean?”  category.  It  includes  understanding  of  enemy  intent,  likely  and 
dangerous  courses  of  action,  and  actions.  It  includes  the  assessment  of  friendly  opportunities  for  favorable  actions  and  the 
associated  risks.  Situation  understanding  also  includes  resolving  and  dealing  with  uncertainty. 

Speed  of 

Command 

Measures  the  time  lag  between  an  occasion  for  action  and  the  implementation  of  action  or  a  decision  not  to  respond. 

Speed  of  Decision 

Measures  the  amount  of  time  it  takes  for  a  decision  to  be  made  beginning  with  the  time  a  need  for  some  action  (or  decision 
not  to  act)  is  identified  through  the  time  where  a  decision  is  made. 

Strike  or  Attack 
Mission  Cycle 
Functions 

Detection,  location,  identification,  decision,  execution,  assessment. 

Survivability: 

data/information 

Measures  the  ability  of  data/information  to  survive  and  operate  in  various  environments.  Measurement  is  a  percent  scale 
(0%=not  survivable,  100%=totally  survivable). 

Survivability: 

network 

Measures  the  ability  of  the  network  to  survive  and  operate  in  various  environments  (at  least  one  complete  path). 

Measurement  is  a  percent  scale  (0%=not  survivable,  100%=totally  survivable). 

Synchronization : 
Actions 

Degree  to  which  actions  are  synchronized.  Metric  is  an  synch  action  level  (0%=no  synchronization,  1 00%=maximum 
synchronization)  or  synchronization  interval  (95%,  90%)  of  actions 
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Synchronization : 
Decisions 

Degree  to  which  decisions  are  synchronized.  Metric  is  an  synch  decision  level  (0%=no  synchronization,  1 00%=maximum 
synchronization)  or  synchronization  interval  (95%,  90%)  of  decisions 

Synchronization : 
Entities 

Degree  to  which  entities  are  synchronized.  Metric  is  an  synch  entity  level  (0%=no  synchronization,  1 00%=maximum 
synchronization)  or  synchronization  interval  (95%,  90%)  of  entities 

Synchronization : 
Plans 

Degree  to  which  plans  are  synchronized.  Metric  is  an  synch  plan  level  (0%=no  synchronization,  100%=maximum 
synchronization)  or  synchronization  interval  (95%,  90%)  of  plans 

Timeliness: 

data  /  information 

Measures  the  utilization  of  the  data/information  as  a  function  of  time.  Metric  is  an  ordinal  scale  (0=no  match  between 
currency  level  needed  and  what  is  available,  10=high  degree  of  matching  between  currency  level  needed  and  available 

Timeliness:  shared 

data/information 

Measures  the  utilization  of  the  shared  data/information  as  a  function  of  time.  Metric  is  a  percentage  scale  (0%=no  match 
between  shared  data/information  needed  and  what  is  available,  50%=  some  degree  of  matching  between  shared 
data/information  needed  and  what  is  available,  100%=high  degree  of  matching  between  shared  data/information  needed  and 
available.) 

Timeliness:  shared 

awareness 

Measures  the  utilization  of  the  shared  awareness  as  a  function  of  time.  Metric  is  a  percentage  scale  (0%=no  match  between 
shared  awareness  needed  and  what  is  available,  50%=  some  degree  of  matching  between  shared  awareness  needed  and  what 
is  available,  100%=high  degree  of  matching  between  shared  awareness  needed  and  available.) 

Timeliness:  shared 
understanding 

Measures  the  utilization  of  the  shared  understanding  as  a  function  of  time.  Metric  is  a  percentage  scale  (0%=no  match 
between  shared  understanding  needed  and  what  is  available,  50%=  some  degree  of  matching  between  shared  understanding 
needed  and  what  is  available,  100%=high  degree  of  matching  between  shared  understanding  needed  and  available.) 

Trust:  Peer-to-Peer 

Measures  the  extent  of  trust  between  entities  that  are  at  the  same  level.  Metric  is  a  percentage  scale  (0%=no  trust,  50%=some 
trust,  100%=significant  trust) 

Trust:  Supervisor- 
to-Subordinate 

Measures  the  ability  of  a  supervisor  to  demonstrate  trust  in  a  subordinate  by  a  willingness  to  delegate  and  allow  subordinates 
to  work  without  constant  supervision.  Metric  is  a  percentage  scale  (0%=no  trust,  50%=some  trust,  1 00%=significant  trust). 

Trust: 

Subordinate-to- 

Supervisor 

Measures  the  extent  of  trust  to  which  a  subordinate  has  with  its  supervisor.  Metric  is  a  percentage  scale  (0%=no  trust, 
50%=some  trust,  1 00%=significant  trust) 

Trust: 

data/information 

Measures  the  extent  of  trust  to  which  an  entity  is  willing  to  rely  on  the  data/information.  Metric  is  a  percentage  scale  (0%=no 
trust,  50%=some  trust,  100%=significant  trust) 

Trust: 

Organization 

Measures  the  extent  of  trust  to  which  an  organization  is  willing  to  rely  on  other  organizations.  Metric  is  an  ordinal  scale 
(0=no  trust,  5=some  trust,  10=significant  trust) 

Trust:  System 

Measures  the  extent  of  trust  of  the  system  by  individuals  and/or  organizations.  Metric  is  a  percentage  scale  (0%=no  trust, 
50%=some  trust,  1 00%=significant  trust) 

Uncertainty: 
shared  awareness 

Measures  the  confidence  level  (0%=uncertain,  100%=certain)  or  confidence  interval  (95%,  90%)  of  shared  awareness 

Uncertainty: 

shared 

understanding 

Measures  the  confidence  level  (0%=uncertain,  1 00%=certain)  or  confidence  interval  (95%,  90%)  of  shared  understanding 

Understanding 

Understanding  is  defined  as  the  process  state  of  drawing  inferences  about  possible  consequences  of  the  operational  situation. 

It  is  based  on  the  ability  of  the  battlestaff  acting  individually  and  collaboratively  to  predict  possible  future  patterns  of  the 
battlespace.  That  is,  whereas  awareness  deals  with  the  battlespace  as  it  was,  understanding  deals  with  the  battlespace  as  it  is 
becoming.  Interpreting  these  patterns  spatially,  functionally,  temporally  in  the  context  of  the  goals/objectives,  constraints,  and 
planned  courses  of  action  envisioned  for  the  operation,  the  battlestaff  begins  to  identify  potential  threats  and  opportunities 
that  demand  a  response  change  or  decision  from  the  command  authorities. 
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Appendix  D:  Subjective  Logic  (Denny,  2010,  Paper  113) 

Subjective  reasoning  is  based  purely  on  one  personal  beliefs,  ideals,  preference  opinion  or 
culture.  For  example  when  you  watch  the  news  and  see  a  story  about  incest,  one  might  be 
offended  at  the  very  thought  while  others  would  not  be  as  shocked  because  it  is  "natural"  for 
their  way  of  life.  Like  some  religious  sects  have  incestual  marriages  and  believe  it  is  right  where 
others  do  not.  These  are  examples  of  subjective  reasoning. 

Subjective  Logic  (Josang,  1997)  (Josang,  2009)  is  a  type  of  probabilistic  logic  that  is  often  used 
in  evidential  reasoning  where  belief  (b),  disbelief  (d),  and  uncertainty  (u)  must  be  explicitly  and 
simultaneously  accounted.  In  contrast  to  systems  described  by  Boolean  Logic,  for  those  systems 
described  by  Subjective  Logic  the  basic  object  is  an  opinion  rather  than  a  fact.  An  opinion  coA(x) 
about  some  proposition  “x”  held  by  source  “A”  is  a  4-tuple  of  the  belief  (bxA),  disbelief  (dxA), 
uncertainty  (uxA),  and  relative  atomicity  (axA)12.  Mathematically,  it  is  not  necessary  to  specify  all 
three  of  the  values;  however,  the  sum  of  the  values  always  equal  one  (bx  +  dx  +  ux  =  1). 

Subjective  Logic  algebra  provides  an  array  of  operations  that  can  be  used  to  manipulate 
opinions.  These  operators  have  many  applications  in  evidential  reasoning  and  data  fusion.  The 
consensus  operator  (written  as)  is  used  for  belief  fusion,  providing  the  capability  to  fuse  possibly 
conflicting  opinions  while  still  forming  coherent,  summary  judgments.  The  underlying 
calculations  on  the  belief  tuple  elements  are  given  in  Figure  D-l. 


Figure  D-l:  Subjective  Logic  Consensus  Operation  (Denny,  2010,  Paper  1 13) 
Subjective  logic  also  provides  a  well  developed  “discount”  operation  (written  as)  that  can  be 
used  for  modifying  the  contribution  of  evidence  based  upon  a  subjective  measure  of  confidence 
in  the  source  of  the  evidence.  The  discount  operator  thus  provides  a  rather  general  means  of 
describing  degrees  influence  and  can  be  used  to  represent  semantic  similarity,  relevance,  trust, 
etc.  The  calculations  for  implementing  a  discount  operator  over  belief  tuples  is  shown  in  Figure 
D-2. 


12  Atomicity  is  the  base-rate  of  the  proposition. 
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l->  A,B  _  u  A  i  B 

bx  -  Db  bx 

a  A,B  _  i  A  i  B 

dx  -  bB  dx 

A,B  i  A  B  t  i  A  B 

ux  =  dB  +  uA  +  bB  ux 

„  A,B _ B 

ax  —  &x 

Figure  D-2:  Subjective  Logic  Discount  Operation  (Denny,  2010:  4) 

An  algorithm  can  be  established  to  measure  the  bias  to  a  situation  as  shown  in  Figure  D-3. 
(Denny,  2010:  13) 


Algorithm  1:  Assign  bias  to  situation  elements: 

For  each  SituationElement,  e: 

Instantiate  Impact  statement,  i,  that  refers  to  e. 

Instantiate  Conviction  statement,  c„  set  to  (the  default)  ignorance  (b — 0.  d — 0,  u=1.0) 
Set  i  to  refer  to  c 

Algorithm  2:  Fuse  propagation  paths  into  Impact  accumulator 
for  each  Judgment  j  (where  the  author  of  j  is  not  “ARID”): 
if  j. about,  is  of  type  SituationElement;  then... 
get  SituationElement  e  to  which  j.  about  refers 
for  each  Propagation  p  that  refers  to  e: 
get  the  Impact,  i,  associated  to  e 
let  cp  be  the  Conviction  of  the  p 
let  ci  be  the  Conviction  of  the  i. 
accumulate  cp  into  ci  by  consensus:  ci  <—  ci  <p  cp 

Algorithm  3 :  Back  propagate  influence  to  supporting  evidence 
for  each  Evidence  statement,  s : 

let  j  be  the  Judgment  that  s  supports 

let  cj  be  the  Conviction  of  j 

let  SituationElement  e  be  evidence  of.? 

let  i  be  the  Impact  of  e 

let  ci  be  the  Conviction  of  i. 

let  d  be  the  strength  of  s 

accumulate  cj  into  ci:  ci  < —  ci  0  (d  Cj) 
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Phases  of  Conflict 

-  Global  Vigilance,  Reach  and  Power  - 
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Network  Centric  Warfare 

-  Operational  Timelines  - 
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Metrics: 

- ->Time  to  Collect  Data 

- ->Time  to  Awareness 

- ->Time  to  Understanding 

- ->  Speed  of  Command 

- -> Speed  of  Action 
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Network  Centric  Warfare 

-  Measures  of  Effectiveness  - 


Network 
^  Centric 
f  Operations 


Network 
^  Centric 
Infrastructure 


National  Level  C 

Measures  of 

Policy  Effectiveness  (MoPE) 

J  President  (NCA) 

Campaign  Level  1 

Measures  of 

Force  Effectiveness  (MoFE) 

j  Joint  Force  Commander  (JFC) 

C2  System  Level  ( 

Measures  of 

C2  Effectiveness  (MoCE) 

y  Joint  Force 

J  Air  Component  Commander 

System  Level 

Measures  of 
Performance  (MoP) 

j)  Theater  Battle  Management 

Core  Systems  (TBMCS) 

Component  Level 

Dimensional 

Parameter 

Joint  Defensive  Planner 

Metrics:  Political,  Diplomatic,  Military,  Economic,  Social,  Information,  Infrastructure 
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Proposed  Nth -Order  Taxonomy 

-  Strategic  -  Tactical  -  Operational  - 


Nth 


Description 


Level 


Area  MOPs/MOEs 


Order 


Actual  Event 

Tactical 

Local 

Desired  Effect 

Tactical 

Local 

DP 

Collateral  Damage  (Physical) 

Tactical 

Local 

DP 

Collateral  Damage  (Non-Physical) 

Tactical 

Local 

DP 

C2  Systems  Effectiveness 

Operational 

Regional 

MoCE 

Command  Intent  Effectiveness 
(e.g.,  JFACC,  JFLCC,  JFMCC) 

Operational 

Regional 

MoFE 

Command  Intent  (JTF)  Effectiveness 

Operational 

Regional 

MoFE 

National  ROE/Policy  Effectiveness 

Strategic 

Global 

MoPE 

Coalition  ROE/Policy  Effectiveness 

Strategic 

Global 

MoPE 

International  ROE/Policy  Effectiveness 

Strategic 

Global 

MoPE 
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Mathematical  Representation 

-  Proposed  Nth -Order  Taxonomy  - 


MOE  (cyber  agility)  =  f(belief)  +  f(disbelief)  +  f(uncertainty) 


Scoring  Metric: 

0.00  =  Detrimental  to  Mission  Operations, 

0.25  =  Unacceptable  to  Mission  Operations, 

0.50  =  Acceptable  to  Mission  Operations, 

0.75  =  Very  Acceptable  to  Mission  Operations, 

1.00  =  Significantly  Acceptable  to  Mission  Operations 


Denny,  Nathan, 

“Mission  Profiles  and  Evidential  Reasoning  for  Estimating  Information  Relevancy  in  Multi-Agent  Supervisory  Control  Applications,” 
15th  ICCRTS,  Paper  113,  June  2010,  page  4,  10,  and  13. 
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Diplomatic  (C4) 
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Conclusions 


•  Need  to  Plan  /  Assess  Operations  within  a  Complex  Endeavor 

•  Combine  PEMSII  and  DIME  into  Proposed  Nth-Order  Taxonomy: 

-  Political,  Diplomatic,  Military,  Economic,  Social 

-  Information,  Infrastructure 

•  Proposed  Nth -Order  Taxonomy 

-  Incorporates  Net-Centric  Operations 

-  Incorporates  Strategic,  Tactical,  Operational  Factors 

•  Provides  consistent  in-depth  analytical  capability 

-  Indicating  7fh-Order  effect  (National  ROE/Policy  Effectiveness) 
will  convey  consistent  context 


Goal  is  to  conduct  robust  analysis  in  developing  COAs 
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